Lecture: Channel coding theorem

Alice wishes to communicate over a noisy channel to Bob. In many engineering problems like wireless networks, the errors are introduced in nature rather than an adversary. The other model where channel is an adversary is also very important, esp. in cryptography. Instead of sending raw message bits, Alice sends a sequence with error correction capabilities so that Bob can counter the noisy behavior of the channel to recover the intended message with high probability. Thus the communication model we are studying is represented in Figure 1. Alice, the sender, first encodes the raw message m of nR bits, i.e. m ∈ {1, ..., 2nR}, bits into a sequence of symbols (x1(m), . . . , xn(m)). Here the channel is used n times. The noisy channel corrupts this sequence into another sequence yn which is received by Bob. Bob then tries to estimate the message m. A rate R is said to be achievable if there are an encoding strategies and a decoding strategies (for each n) so that P(M̂ 6= M) → 0 as n → ∞. The capacity of the channel is the supremum of all the achievable rates.